Computer vision systems are systems that analyze camera generated images and enable useful conclusions to be reached from that analysis. Such systems often comprise digitizing an analog image to produce a table of digital data. Thereafter, a digital computer processes the digital data, for example, to locate edges, recognize shapes, or to verify spatial relationships. Eyetrackers are a splendid example of computer vision systems. An eyetracker is a device that measures the movements of the eye upon which the camera is trained. Specifically, an eyetracker detects the direction an eye is gazing. An eyetracker communication system allows the physically handicap person to use eye gaze and movements to communicate. Examples of computer vision systems particularly useful as an eyetracker are set forth in U.S. Pat. No. 4,648,052 entitled "Eyetracker Communication System" and allowed U.S. patent application Ser. No. 06/897,497 entitled "Computer Vision System Based Upon Solid-State Image Sensor." Both of these patent documents disclose an eyetracker wherein the direction of eye gaze is determined by comparison of the location of the center of the image of the corneal reflection and the location of the center of the image of the pupil. While many of the concepts and techniques disclosed in those patent documents are used in the various embodiments of the invention disclosed herein, a drawback in those systems has been overcome. A simple technique is provided for determining the orientation of an object the image of which has been generated by the computer vision system. This technique has application to body pointing in general and robot pickup systems. It has resulted in a substantially improved eyetracker in which it is no longer necessary to locate the image of the pupil.
An eyetracker (eye gaze sensing system) or almost any other computer vision system is improved if the field of view and depth of view of the image are enlarged. In the case of an eyetracker, the larger the field of view, the easier it is to position the device such that the user's eye can be viewed. Maximizing the depth of the field of view is particularly important since it allows the user to be at various positions along the optical axis. One technique for improving the depth of the field is to use an automatic focusing lens, for example, a lens that will mechanically move in order to focus the image of objects at various depths along the optical axis. A mechanical focus is not often desirable. It adds to the expense of the system, adds to the physical size, decreases reliability due to mechanical wear and tear, increases the audible noise generated by the system and slows the eye gaze sensing process during the time when focusing is taking place because the image is blurred at that time. An alternative to an automatic mechanical focusing lens system is to use a fixed focus lens system; that is, a lens focusing an image of an object at only one depth. The depth of field inherent in the fixed focus system is a function of the f number of the lens. The higher the f number, the smaller the diameter of the lens. Also, the higher the f number, the less the capacity to gather light and the larger the depth of field. To maximize the depth of field, the f number must be maximized but this results in less light being gathered by the lens and focused on the image. Hence, to maximize the depth of field by increasing the f number, it is essential to increase the brightness of the image features to be analyzed.
Previous eye gaze sensing techniques have used the position of the corneal reflection relative to the center of the pupil to determine eye gaze location. Because the pupil is very dark, lenses with a low f number have had to be used resulting in an eye gaze system with a small depth of field. To alleviate this problem, the applicants have developed an eye gaze sensing technique that utilizes only bright reflections. The image of the pupil is ignored, thus enabling the use of a lens with a high f number resulting in a large depth of field.
A feature of the applicants computer vision system is a simple technique for determining the three-dimensional (3D) position and orientation of a plane in the field of view. In the case of an eyetracker according to this invention, it is used for determining change in the user's head position.
One prior technique used for determining the position and orientation of an arbitrary plane surface of the object is to view the object with two cameras. Corresponding features of the two images are then matched and a triangulation function is used to determine the distance of the feature points from the cameras. See IEEE PAMI article "Error Analysis in Stereo Determination of 3-D Point Positions", Nov. 1987 and the extensive bibliography of stereo camera literature. Three feature points on a plane serve to uniquely specify the orientation of the plane surface.
Two cameras increase the cost of the system. Therefore, it is desirable to determine the 3D position and orientation of a surface given only one view. This is possible if certain a priori information about the image geometry and behavior of the object to be viewed is available. The image processing literature presents algorithms for extracting 3D information from single perspective views. See, for example, IEEE PAMI article "New Methods for Matching 3-D Objects With Single Perspective Views", May 1987 and the article bibliography. These algorithms are generally concerned with looking at real world scenes and are very computationally intensive. Some of the algorithms use geometric models generated from a CAD data base in order to recognize and locate the corresponding parts in 3D with the machine vision system. These also tend to be very computationally intensive.
According to this invention, a unique reflective patch is attached to the surface whose orientation is to be determined. This results in a simple and inexpensive way to detect 3D orientation and position. In addition, the use of the reflective patch as described herein would reduce the computational complexity of algorithms using two views.